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Abstract 

Commercially developed basal reading programs are used in most 
elementary school classrooms in this country. Yet, often neither 
the publishers developing these programs, nor the members of 
state and local textbook adoption committees selecting programs 
are able to take advantage of the best available and most up-to- 
date knowledge about the reading process and reading instruction. 
The development and piloting of A Guide to Selecting Basal 
Reading Programs is a major effort to make such knowledge 
available to publishers and members of adoption committees. The 
Guide presents current information about research and practice in 
reading, and provides guidelines for evaluating basal reading 
programs. Case studies of four textbook adoption committees that 
piloted the Guide are presented. In general, these committees 
found the Guide contributed to a more informed selection process, 
and committee members focussed more attention on substantive 
issues associated with the quality of instruction and content, 
and less attention to superficial aspects of programs and 
political considerations. In particular, four factors 
contributed to the success of the project: committee leadership 
and enthusiasm, adequate time to use the materials, and iuservice 
support from a knowledgeable source. The role of this project in 
the larger endeavor to improve reading instruction in our 
elementary schools is addressed. 
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Improving Basal Reading Programs: 
A Report of the Adoption Guidelines Project 
at the Center for the Study of Reading 

The Center for the Study of Reading's attempt to improve 
basal reading programs- -and the textbook adoption process- -is 
described in this paper. The Adoption Guidelines Project at the 
Center has been engaged in developing a series of booklets on 
topics important to reading instruction. The goal of the project 
is that the booklets in the series, A Guide to Selecti ng Basal 
Reading Programs, will be a catalyst for change and improvement 
in reading instruction. This goal is premised upon the belief 
that members of textbook adoption committees who are more 
knowledgeable will not only be better at making decisions, but 
will demand better basal reading programs from the publishers. 

In this paper we first briefly document the use of basal 
reading programs, discuss the improvciment of these pro^jrams, and 
review the research about the process of textbook adoption. We 
then discuss how the Adoption Guidelines Project group developed 
the booklets- -and especially how tryout of the booklets with 
textbook committees shaped and altered their content. We 
describe the booklets in their final form, and finally, we 
summarize case studies of the adoption process of four school 
districts that used the booklets. 
The Use of Basal Reading Programs 

The predominance of basal reading programs in American 
school classrooms is well documented (Austin & Morrison, 1983; 
Shannon, 1983; Hoffman & O'Neal, 1984; Duffy & Mclntyre, 1982; 
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Borko, Eisenhart, Kello, & Vandett-. 1984; Mason & Osfcorn, 1982; 
Durkin, 1978-79). Classroom : r- i has established that both 
the content and quality of it^st/ n in basal readers 

influences reading curriculun and ruction in American 
schools. The estimates are that 7:3 90% of the time students 
spend in a reading period is dominated by the content of basal 
reading programs (Fisher, Berliner, Filby, Marliave, Cohen, 
Dishaw, & Moore, 1978; Mason & Osbo a, 1982). Some researchers 
have found that teachers follow the teachers* manuals of basal 
programs quite closely (Hodges, 1980; Durkin, 1983; Woodward, 
1986). It is for these reasons that many scholars, including 
Brophy (1982) and Resnick and Resnick (1985) argue that improving 
basal reading programs is a necessary condition for improving 
reading instruction in American classrooms. 
Tlie Improvement of Basal . Reading Programs 

It is only during the past decade that the content and 
quality of basal reading programs have been extensively 
investigated by a number of researchers in different universities 
and research institutions. In an effort to communicate some of 
this research to the publishers of basal reading programs, the 
Center for the Study of Reading and the Educational Division of 
the Association of American Publishers have sponsored three 
important meetings. At these meetings, which have taken place 
over the past six years, executives and editors of all the major 
publishing houses in the United States and Canada listened to 
presentations of reports of research about reading. At each 
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meeting the emphasis was on research that had direct implications 
for change in basal reading programs.^ 

As the executives and editors discussed these reports (in 
prepared responses, panel discussions, and privately), a 
recurring theme emerged: the impetus for change must come not 
only from publishers, but also from the purchasers of basal 
reading programs - -that is, the school districts buying programs, 
and most specifically, the teachers and school administrators 
responsible for selecting the programs. The point was made (in 
one form or another, over and over again) that changes in reading 
programs must be perceived as important and necessary by the 
people who buy programs; only then would most publishers be 
willing to take the financial risk of substantially altering 
programs to include procedures and ideas supported by recent 
research in reading. 

Evolving from this theme was a challenge to the Center to 
communicate current reading research (as well as exemplars of 
good practice) to a targeted group of people: teachers, 
administrators, and other members of textbook adoption 
committees. Underlying this was the belief that a more 
knowledgeable market would demand- -rather than reject- -reading 
programs that contained the kinds of change advocated by 
researchers. 

In response to this challenge, researchers at the Center 
began developing a series of booklets, A Guide to Selecting Basal 
Reading Programs . The plan was for the booklets to describe and 
summarize recent research in reading and to present implications 



Adoption Guidelines Project - 6 

of that research for classroom instruction- -especially for the 
content of basal reading programs. 

It was decided that researchers at the Center for the Study 
of Reading, as well as researchers from other institutions, would 
write the booklets and that teachers and school administrators 
would be consulted on the project. It was also decided that, as 
the booklets were developed, they vzould be cried out and 
evaluated by textbook adoption committees in school districts. 
The three authors of this paper worked as the Adoption Guidelines 
Project staff whose task it was to coordinate these development 
and tryout efforts. 
The Textbook Adoption Process 

Project staff first investigated what research had to say 
about the textbook adoption process; for example, how are 
conijnittee members selected? How do they establish criteria? And 
how do they make decisions? Our search through the literature 
revealed that until recently the textbook adoption process was 
not studied in a systematic way. Over the last few years, 
however, Farr and his colleagues (Courtland, Farr, Harris, Tarr, 
& Treece, 1983; Farr & Tulley, 1985; Fan:, Tulley, & Rayford, 
1984; Powell, 1985; Tulley, 1983) have conducted a series of 
studies that provide some data about the textbook adoption 
process. While conventional wisdom suggests that committees will 
choose the basal program of the highest instructional quality, 
these researchers found that when evaluating programs, committees 
tjrpically used checklists to determine instructional quality. A 
review of these checklists by Farr and Tulley indicates that 
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"most [checklists] emphasize only the existence of a particular 
factor rather than its quality" (Farr & Tulley, 1985, p. 470). 
For example, a mark on a checklist can indicate that main idsa 
instruction is present but does not indicate the amount or 
quality of the instruction. 

Courtland, Farr, et al. (1983) reported that the evaluation 
criteria employed by most reviewers in their study were based on 
"personal interpretations" of the general criteria that were 
supplied to them. They wrote: 

When asked specifically to identify the factors that 
caused them to select as "best" one set of textbooks, 
the reviewers responded with a wide variety of general 
information and little specificity. The researchers' 
general impression was that the reviewers were often 
looking for reasons to exclude a set of texts rather 
than significant reasons to adopt one textbook series 
(p. 76). 

An item analysis of basal reading evaluation forms and 
checklists gathered from 26 ijchool systems in 14 states (Comas, 
1983) revealed few items that require committee meaibers to 
document or substantiate in a quantifiable form any of their 
conclusions about a program. 

We began our own collection of checklists and rating forms. 
As we examined them, we concluded that the very nature of these 
lists and forms precluded objectivity: answering yes/no 
questions about sufficient review, or rating on a scale from 
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1 to 5, how well teachers' guides are organized, are measures 
that will yield biased, rather than objective, responses. 

In addition, the lack of representation of research-based 
criteria on the forms and lists was evident. They typically 
contained more items about the physical features of textbooks 
than about more substantive topics such as specific content, 
prose style in readers, pedagogy, ^^d instructional design in 
teachers' guides. And, in fact, the Comas item analysis of 
evaluation forms revealed that only 34.6% of the districts 
sampled included research questions of any kind on their 
evaluation forms, as compared to 73.1% that included items about 
illustrations. 

After reviewing research on the textbook adoption process, 
we determined a major goal of the project: to help textbook 
adoption committees focus on the content of basal reading 
programs and the quality of instruction. We hoped the b -oklets 
would (a) help textbook adoption committees understand the issues 
involved in effective reading instruction, (b) describe what 
committees should look for when evaluating the content of basal 
reading programs, and (c) provide a procedure that would help 
committees both to effectively analyze the content of programs 
and to record their findings. We also hoped the booklets would 
provide textbook adoption committees with knowledge with which 
they could make a case to their publishers for improved reading 
programs . 
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Development of the Booklet s 

From the beginning, the project staff worked with textbook 
adoption committees. Committee members indicated they wanted 
current information about effective reading instruction. They 
also expressed a need for precise ways of examining basal 
programs in order to fii.d out how instruction was handled in 
different programs. It seemed that traditional checklists were 
not enough, information about research and practice was not 
enougu, and even guidelines were not enough. One reading 
coordinator of a fairly large school district summed up the 
requests of the teachers in her district: "Tell me how I should 
go about looking for these things in the programs I evaluate." 

Such requests guided the initial efforts. Our hope was that 
committee members would read and make use of the information in 
the booklets. Two questions predominated: (a) What information 
would committee members find most useful? and (b) How should the 
information be presented? We were uncertain about how the 
booklets would look or what to do to help adoption committees 
examine programs. We realized that the booklets could not be too 
lengthy and that they should nit be laden with research jargon. 
We also realized that they had to include evaluation procedures 
more substantive than the typical checklists and rating forms. 

These ideas directed the writing of early drafts of the 
booklets, which were then tested with several adoption 
committees. These early drafts had many problems: They were too 
long, too difficult, and full of jargon. The committees found 
the evaluation procedures cumbersome and confusing. Some of the 
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problems the staff encountered and the solutions to these 
problems are discussed below. 

Booklet length. A member of one adoption committee 
requested that each topic be summarized on one 8 1/2 by 11 inch 
piece of paper. In contrast, Project staff envisioned short, 
five-page pamphlets on a number of important topics. The early 
drafts, however, were from 25 to 30 pages in length. In their 
concerns for completeness, Center staff members working on the 
booklets were reluctant to shorten these drafts. Pilot tests 
showed that these 25 -page booklets were not very successful. For 
example, members of two different committees agreed to read two 
booklets; yet a month later many of the booklets were returned 
unopened. Center staff members responded to this real-world test 
by cutting the length of the booklets to about 15 pages. 

Booklet format. Members of the editorial staffs of some 
basal publishing houses who reviewed the booklets, as well as 
committee members, found the format of the early drafts difficult 
to follow. These complaints led to a major rearrangement and 
reorganization of the information in the booklets. 

Amount of work . Members of several committees commented on 
the inordinate amount of work involved in using early drafts of 
the booklets and especially the evaluation procedures. One 
committee read one booklet and, instead of using the booklet's 
procedures, made up its own checklist. Another committee decided 
against using the booklets and, instead, used copyright dates of 
the programs it was examining to make their initial selection. 
Still another committee resorted to the "flip test." Such real 
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world tests prompted Project staff to revise the procedures and 
reduce considerably the amount of time needed to complete them. 

Content of booklets. Committee members were especially 
helpful in focusing Project staff on teachers' concerns. Where 
possible, these concerns were addressed in the booklets. For 
example: How many vocabulary words should be taught before a 
selection is read? What kind of questions should be asked about 
a selection? Sometimes the staff decided the booklets could no- 
address teachers' concerns simply because there was not enough 
information available to make statements about the value of an 
activity or topic, or even to make an argument about usefulness 
of the activity. For example, to the question "Should skill 
teaching appear in the student textbook?," there is no good 
answer . 

Evaluation procedures. One of the most controversial 
aspects of the booklets was the inclusion of procedures for 
evaluating the treatment of different topics in reading programs. 
Many of the consultants to the project from research institutions 
said that it was sufficient to present information about current 
research and practice in reading, and that the development of 
evaluation procedures was beyond the scope of the Project. 

Textbook adoption committees, however, strongly disagreed. 
In fact, every adoption committee we worked with advised us to 
include evaluation procedures. Otherwise, they said, the 
booklets would not be used. Committees wanted simple directions 
for what to look for, and advice for how to evaluate what they 
found. The procedures in early drafts of the booklets were 
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cumbersome and lengthy. Eventually, after much trial and error, 
procedures were devised that permitted committees to examine 
instruction in the programs they were evaluating and to 
systematically record their comments on worksheets. 

In summary, we are convinced that the information we 
obtained from the textbook adoption committees using the various 
versions of the booklets is important. These experiences also 
impressed us with the difficulty of our task. Project staff 
tried to find a middle line between general and specific 
information and between too much information and not enough. The 
content and organization of each booklet was changed over and 
over again. Quite literally, dozens of drafts were tried out 
with different textbook adoption committees for over a year. 

We became convinced that committees were not willing to read 
lengthy, theoretical essays about the reading process. Rather, 
they wanted concise information about current research and 
practice and then assistance in applying that information to 
basal program evaluation. We found that while committee members 
were often desperate for help in learning how to evaluate 
textbooks, they wanted a specific kind of help. 

The content and organization of the final versions of the 
booklets are the result of these efforts. They are described in 
the next section. 
Description of the Booklets 

^ G^^^Q to Selecting Basal Reading Programs consists of 
seven booklets and a Leader's Manual . Each booklet discusses a 
specific topic important to reading instruction: 
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• Beginning Reading and Decoding Skills — issues related to 
early reading (K-2) and decoding skills such as phonics, 
contextual analysis, structural analysis, and syllabication. 

• Selection in the Reader - -the content, quality, and 
difficulty level of the student readers. Issues related to 
the readability of textbooks are also discussed here. 

• Comprehension The Guided Reading Lesson- -what the teacher 
does to guide students' rp.ading in the student readers. The 
booklet examines in detail the guided reading lessons that 
are provided in the teachers' manuals of basal reading 
programs . 

• Comprehension II: Skills and Strategies - -comprehension skill 
instruction. This booklet discusses important new research 
about improving students' comprehension abilities and also 
addresses some age-old issues and dilemmas about 
comprehension skill instruction. 

• Workbooks --the quality of instruction included in workbooks, 
worksheets, ditto masters, and other workbook type 
materials. The booklet discusses how workbooks are used in 
classrooms and shows how to analyze the content and format 
of workbooks. 

• Assessment, Management, and Software --the tests, management 
systems, and software that are included in many basals. The 
booklet presents research about assessment issues and 
discusses how to evaluate management systems and software. 
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Each booklet is divided into four sections. The first 
section (Research and Practice) discusses current research and 
practice about the booklet's topic. The second section 
(Guidelines) lists the guidelines that emerge from the research 
and practice section. The third section (Procedures) contains 
directions for using the worksheets to record comments and 
evaluations. The last section (References) supplies references 
for further reading and study (see Table 1). 



Insert Table 1 about here. 



The Vocabulary Booklet was developed first and served as a 
prototype for the rest of the booklets. We will describe this 
booklet in detail to give a sense of the organization and content 
of all of the booklets. 
The Vocabulary Booklet 

In the research and practice section of the Vocabulary 
Booklet, textbook evaluators are provided with summaries of 
current research about vocabulary acquisition and instruction. 
The section begins with a general introduction to issues in 
vocabulary acquisition. Evaluators are reminded that knowing the 
meanings of words is not an all-or-none situation, and that there 
are different levels of word knowledge. Research is presented 
to demonstrate, for example, that it takes extensive word 
knowledge to improve reading comprehension and that having a 
limited knowledge of a word (such as knowing its definition) is 
not likely to improve comprehension. It is established that 
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vocabulary instruction must lead to extensive word knowledge if 
it is to affect reading comprehension. 

Three major approaches to the teaching of vocabulary in 
basal reading programs are described and evaluated: 
definitional, contextual, and conceptual. How each approach can 
lead to different levels of word knowledge and affect 
comprehension is discussed, and examples of -ore and less 
effective instruction are provided. 

As each approach is explained, a guideline is presented. 

For example, a guideline about what to look for when evaluating 

the contextual approach follows a discussion of that approach: 

When a basal program uses a contextual approach, 
look for: 

• instructional contexts in which sentences provide 
enough context clues for students to accurately 
figure out the meanings of words. 

Following the research and practice section is a list of all 

the guidelines that appear in the research and practice section. 

See Table 2 for the Vocabulary Guidelines. 



Insert Table 2 about here. 



EKLC 



The procedures follow the guidelines, which show evaluators 
how to determine how well the guidelines are met in a program. 
Evaluators record their findings on a worksheet. The Vocabulary 
Worksheet (see Table 3) provides space for comments on the 
vocabulary activities in a given lesson. Evaluators analyze 
which approaches the program uses and, for each approach, make a 
judgment about the quality of the instruction. For example, when 
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a program uses a contextual approach, evaluators must ask, "Do 
the sentences provide enough context clues for students to figure 
out the meanings of words?" Comments are recorded in the 
appropriate spaces; additional space is provided for ratings. 



Insert Table 3 about here. 



The Effectiveness of the Booklets 

The examination of basal reading programs is part of a 
larger textbook adoption process. Farr and his associates (Farr 
& Tully, 1985; Powell, 1985) describe textbook adoption as a 
dynamic group process that has its own rules, agenda, and 
organizational structure. The piloting of our booklets 
demonstrated to us that the dynamics of the process influenced 
the extent to which the booklets were used by textbook adoption 
committees . 

Center staff followed four adoption committees that used the 
final version of the booklets to help them select basal reading 
programs. Data were collected from observations, audiotapes, 
questionnaires, and interviews? with committee members. The focus 
of the present discussion is on the question: Under what 
conditions will the booklets work? That is, under what 
conditions will the booklets help adoption committees evaluate 
the content and instructional quality of basal reading programs. 
Complete data are not presented here (see Dole, Rogers, & Osborn, 
in preparation). Instead, our purpose is to summarize our 
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experiences and identify some issues that we think need further 

exploration and study. 

A Description of the pilot Sites 

Site A. The first textbook adoption site studied was 
located in a suburb of a major city in the Midwest. The adoption 
committee consisted of seven teachers representing each of the 
seven schools in the district and was led by the reading 
coordinator of the district. 

The Site A Committee used the booklets to choose a basal 
program from among the three series that had already survived 
several cuts. Committee members broke into groups of three. 
Each group read one booklet (either the Vocabulary , W orkbook or 
Comprehension I booklet). The groups then met with the Adoption 
Guidelines Project staff, who showed them how to use the 
worksheets. Group members worked individually and also met 
periodically, when the groups completed their work, the 
committee met as a whole. Each group presented an oral report 
summarizing how the topic of ir.s booklet was handled in each of 
the three programs. Then the entire committee rated and ranked 
each program on each of the booklet topics. 

Site B. The second textbook adoption site studied was 
located in a western agricultural town. This school district 
viewed the textbook adoption process as part of a larger change 
effort to improve reading instruction. The district had embarked 
on a year « long staff development program that was coordinated 
with choosing a new basal reading program. A group of 50 
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district personnel (including a principal and a teacher from each 
school in the district) participated in this effort. 

From among those principals and teachers, seven were chosen 
as team leaders to help train and work with the rest of the 
group. The Adoption Guidelines Project staff trained this 
smaller group in the use of the booklets; then they led the rest 
of the committee members through the evaluation process. The 
committee broke into small groups; each group evaluated the three 
topics in several basal reading programs. 

The committee has completed its evaluation of vocabulary, 
comprehension instruction, and the workbooks in the basal 
programs they are considering. At the time this article is being 
written, the committee is still in the process of evaluating 
other topics. 

Site C. Adoption committee C was located in a small 
Midwestern town. In September, eighteen committee members were 
given the booklets which they used individually as they piloted 
five different reading programs. Each of the committee members 
piloted two programs with two different reading groups. Monthly 
meetings were scheduled and committee members met by grade levels 
to compare notes and to come up with evaluations and ratings of 
each topic for each program based on the worksheets aad on the 
piloting. In January a decision was made. Two basals were 
chosen with an option for one school to use a third. 

Site D. The final pilot site was located in a small town in 
the Midwest. The committee consisted of 25 volunteer teachers 
from the district, and was led by the district reading 
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coordinator. This district took six months to complete the 
adoption process, and, like Site C, used the three Adoption 
Guidelines Booklets to evaluate the series they wore 
simultaneously piloting. 

This adoption committee, like the Committee at Site C, did 
not have any training from Project staff as committee members 
evaluated the program. But, unlike Committee C, the Site D 
leader had no support or assistance from Center staff. The 
booklets were sent to the committee which met as a group with 
their leader to ccsplete the worksheets together. 
Discussion of the Four Sites 

A comparison and contrast of the four sites raises 
interesting issues concerning the effectiveness of the Adoption 
Guidelines Booklets. One goal of the project was to help 
committee members focus on the content and instructional quality 
of basal reading programs. Our pbservations and audiotapes of 
committee meetings indicated that the four committees looked at 
programs in considerable detail. Teachers at all sites completed 
lengthy written evaluations on the worksheets and gave very 
specific examples from basal programs to support their findings 
and conclusions. 

Field notes from observations of committee meetings 
illustrate how the booklets helped adoption committees focus on 
the instructional content and quality of basal programs: 
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From Site B evaluating vocabulary instruction: 
Committee 

Member: This text uses a co.nceptual approach, but it 

is not labeled. The text is dependent upon the 
teacher, I would like to see more in the manual 
for those teachers who don't add information if 
they don't know it. What will only be taught^^s 
what is in the manual . if^ 

Project 

Staff: Can you give me an example? 

Committee 

Member: Plant. One second-grade story is about a plant 

[a factory] . . . Plant is later discussed in 
comprehension, but not directly. Children are 
not introduced to the concept of a plant as a 
factory, though that's what the story is about. 

From Site A evaluating comprehension instruction: 

These are not good questions. They are wide 
open, too general. The whole series of 
questions are never clarified; there's a whole- 
barrage of questions to ask and no directions 
for teachers . 

From Site C evaluating workbooks: 

The language used in the textbook is also used 
in the workbook. Students know what they are 
to do and why they are to do it. Directions 
are clear and enable students to do the 
workbook (page) independently. The pictures 
are precise and realistic . . . directly related 
to the task . . . uses a variety of response 
forms . . . very clear instructional language. 

An incident at Site A points out nicely that some committee 

members need assistance before they can begin to critically 

evaluate the quality of the instruction in basal reading 

programs. The committee members' hesitation to find fault could 

well reflect Shannon's (1983) assertion of the "reification of 

the basal." That is, teachers believe that the basal is "Bible," 

and that they are not in a position to evaluate the quality of 



Committee 
Member: 



Committee 
Member: 
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the instruction in a basal reading program. During one session, 
a member of the Project staff found that teachers who were 
evaluating comprehension instruction were baffled as they read 
over 30 questions following a lesson they were evaluating. 
Project staff: What seems to be the problem? 



Committee 
Member: 



Project staff: 



Project staff: 



Project staff: 



Committee 
Member 1: 



Project staff: 



Committee 
Member 1: 

Project staff: 



Committee 
Member 2: 



This lesson has over 30 questions about the 
story. But the comprehension worksheet for 
the Comprehension Booklet has space for only 
12 questions. 

Well, what do you think about having 30 
questions for one story? 

(No response. Committee Member 1 looks 
at Committee Member 2. Committee Member 2 
looks back at Committee Member 1 and shrugs 
her shoulders). 

Is that good or bad? 

(No response.) 

Well, ^take a look at this story. If you were 
teaching this story to your kids, would you 
ask all of these questions? 



Well, no, I wouldn't. There are too many 
questions to ask kids. It would ruin the 
story for them. 

OK. So now what do you think of this program^s 
handling of questions for this grade level? 
Do they do a good job or not? 



Well, no, I guess they don't. 

Do you think your own questions would be 
better in this particular case? 



Yes, I guess so. But I never would think to 
question the way the basal would do it. 
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Project staff: Well, now you see that you can and you should. 

Now you need to see if this is a particular 
pattern that runs through this whole basal 
program. If so, then youVe ^lad^i an evaluation 
about the way one basal handles comprehension 
questions . 

> 

The Project staff member followed this group's progress for 
the next hour. During that time the group evaluated three 
additional lessons in the same basal program and then two lessons 
in another program. Their conversation was lively and animated. 
They decided that the second program did a better job of asking 
questions than did the first program. But they also agreed that 
neither program provided the questions that they would like to use. 

In this paper we present only a partial analysis of the 
observations, audiotapes, questionnaires, and interview data we 
collected; however, two general impressions pradominate. First, 
the booklets "took the guesswork" out of evaluating basal programs. 
One teacher said, "In the past there has been too much, 'I feel,' 
which is totally subjective." Another teacher said, "Previous 
attempts at adoption centered around debate based on opinion and 
who screamed loudest. This time it has been systematic, factual 
debate." The leader at Site A reported that her committee "could 
evaluate basals thoroughly. They knew the programs inside out 
when they were done. They quantified their evaluations and 
justified their responses." Leaders at each of the sites 
corroborated these general feelings. 

Our second impression was that after committee members 
completed their evaluations, they found that no basal program 
really matched the quality of instruction that they had been 
looking for. The following remarks are sobering: 
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• The booklets gave us an opportunity to look closely at 
the programs we were evaluating. In the end, we chose 
the best of the worst. 

And another: 

• It*s disappointing to do all this work and then find 
that there ar^ not really any good programs out there. 

The leader from Site A reported: 

• The tendency is to see all basals as poor after they go 
through the procv^,ss« -like the adolescent who finds out his 
parents are not perfect . . . 

Such comments made it clear to us that the committees felt 
that the basal programs could not match the high instructional 
standards set by the booklet guidelines. And, because the basal 
programs fell short of their expectations, committee members saw 
the need for staff development once the adoption process was 
complete. "In the end, in-servicing becomes important," said one 
committee member. The leader at Site B reported, "My teachers 
said to me, 'How do I tell other teachers this*?" 
Factors That Lead to the Booklets* Effectiveness 

We have ample evidence that committees at all four sites 
did, in fact, carefully evaluate the instructional quality of the 
basal programs they examined. Data from the sites indicated that 
committee members spent considerable time examining programs and 
that their final choices reflected their best judgments about the 
quality of instruction in those programs. 

Nevertheless, our observations do not lead us to believe 
that the booklets will be used successfully by all adoption 
committees. On the whole, reactions from committees at Sites A, 
B, and D were positive: Committee members felt that the booklets 
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helped them evaluate instructional quality and thus to do a 
better job of selecting programs for their districts. Committee 
members at Site C, however, had mixed reactions to the booklets 
and to their usefulness in helping them select a basal program. 
Data from these four sites have led us to speculate about factors 
that influence the effectiveness of the booklets. These factors 
are discussed in the following paragraphs. 

Time. Many committee members commented on the enormous 
amount of time involved in using the booklets. The committee at 
Site A agreed that because of the time factor, the process was 
"not for everyone." Time had come up as an important factor 
influencing the decisions of other adoption committees (Farr & 
Tulley, 1985). Courtland et al. (1983) found that a major 
concern of the adoption committee they studied was that committee 
members perceived themselves as not having enough time to review 
the materials adequately. 

Another aspect of time was the presence or absence of 
release ti.me. Committees at Sites A, B, and D were given release 
time to work on their committees. Site C received no release 
time, and, as one teacher reported, they were "expected to 
volunteer. " 

Enthusiastic committee members . Committee members at Site A 
also commented on the importance of an interested, enthusiastic 
committee. They said that such a committee would be needed to 
sustain the time and effort involved in looking closely at basal 
reading programs. This committee appeared to have that 
enthusiasm. At least three of the members were involved in 
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gradtiate classes in reading, and the leader reported that they 
all "enthusiastically" volunteered for uhis adoption committee 
assignment. Committee members at Site B were by and large an 
enthusiastic group as well. Most teachers were "hand-picked" by 
their principals as master teachers. The subgroup of seven 
leaders had been chosen because of their leadership abilities and 
their enthusiasm for the reading inservices provided by Project 
staff. 

On the other hand, the Site C committee did not appear to be 
as interested and enthusiastic as the other committees. This was 
evidenced by their arriving late to after-school meetings, lack 
of questions, inattentive behavior during sessions, and failure 
to complete our questionnaires. 

Inservice. Committees at Sites A and B had extensive 
inservice training in reading, along with training in the use of 
the booklets. Individual interviews with members of both 
committees indicated that it was their combined knowledge about 
reading instruction, rather than knowledge gained from the 
booklets alone, that led to their understanding of the material 
and their ability to use the booklets to evaluate basal programs. 
When Project staff interviewed individual committee members about 
their current thinking about the use of basal programs, they all 
reported changing something in their teaching or their use of 
basal programs. But, they all pointed to graduate classes, 
readings, or inservice training, as well as the booklets, as 
sources of their current thinking. 
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On the other hand, the Site C committee had no related 
inservice training or readings and no assistance in how to use 
the booklets. As a matter of fact, this committee did not work 
together at all. They were given the booklets, told to read 
them, and then to use them to evaluate the programs they were 
piloting on an individual basis. 

Committee D did not have any Project staff assistance 
either, but they did have a knowledgeable leader who knew reading 
research and practice, and who provided additional readings and 
assistance to the committee. 

Leadership. An additional factor that seems to strongly 
influence the perceived usefulness of the booklets is the kind 
and quality of the leadership available throughout the textbook 
adoption process. Committees A and D had knowledgeable reading 
coordinators taking strong leadership roles in the adoption 
process. They were decisive in assigning duties and deadlines, 
and they were able to answer questions about reading content. 
And when an issue needed clarification, they were able to give 
it. 

Committee B also had a strong leader. Although he was not 
knowledgeable about reading, he utilized the services of Project 
staff to provide that leadership role for the adoption committee. 
So, when committee members needed clarification about the use or 
content of the booklets. Project staff were there to help. 

A change of leadership midstream caused organizational 
problems for committee C, and neither leader was perceived as 
particularly effective. As one committee member commented, "[the 
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first leader] was very opinionated and [the second leader] was a 
nice person but not demanding." 
Summary , Conclusions and Caveats 

The goal of the Adoption Guidelines Project is the 
improvement of basal reading programs. The focus of this article 
has been on materials and development to help textbook adoption 
committees evaluate the instructional quality of these programs. 
We described the development of the Adoption Guidelines booklets 
and discussed data collected to determine if the booklets were 
useful. We have preliminary evidence to suggest that, under 
certain conditions, the booklets are quite effective in helping 
adoption committees evaluate the content and instructional 
quality of basal reading programs. 

We identified four factors that seemed to influence the use 
of the booklets: time, enthusiasm, inservice training, and 
effective leadership. Whether any or all of these factors is 
necessary and/or sufficient is a question for further study. 
Other questions remain as well. For example, how will the 
booklets fit into the total textbook adoption process? 

Powell (1985) identifies five categories of factors that 
influenced the selection decision in the adoption committee she 
interviewed: (a) politics- -the personalities of the people on 
the committee; (b) publishers- -the publishers' representatives 
who give presentations about their programs; (c) pilot tryouts-- 
the length and quality of the pilot studies; (d) processes --the 
organization and group process of the committee; and (e) 
pedagogy-. the feelings of the committee members about reading, 
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and about current and local educational trends. It is 
interesting to speculate about how the use of the booklets fit 
into these categories. 

(a) Will use of the booklets influence the politics of the 
committee? At two sites, comments were made by committee members 
indicating that with the booklets the evaluations became less 
"subjective. " 

(b) What effect will the booklets have on publishers? 
Although several committee members suggested that the booklets 
should be read by all publishers and their representatives, we 
have no data on this topic. 

(c) How will the booklets interact with pilot tryouts? We 
have some evidence to suggest that members of two committees 
perceived the combined use of piloting and the booklets as being 
hel:-ful. 

(d) Will the booklets affect the textbook adoption process 
itself? The booklets provided additional structure for 
committees. We observed that some committees particularly 
appreciated the structure built into the booklets, but we do not 
know to what extent this structure will be used. 

(e) Do the booklets affect the amount of emphasis on 
pedagogical factors in adoption committees' decisions? We have 
some evidence that the booklets helped committees to focus on the 
instructional quality of basal programs in a way that was not 
possible with checklists. One of the most consistently heard 
comments made by committee members was that the guidelines gave 
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specifics to look for in evaluating each program. One teacher 
said, "I wouldn't have known where to begin on my own." 

A more global question remains. What place do the Adoption 
Guidelines Booklets have as an agent for improving the quality of 
reading instruction? Most committee members who used the 
booklets reported that they became more knowledgeable about 
reading instruction. They also reported that they became more 
aware of the importance of the new knowledge for themselves and 
for their colleagues. And, as they realized the inadequacies of 
basal programs they became aware of the need for more inservice 
training and staff development in their schools. 

Finally, will the Adoption Guidelines Project play a role in 
improving the quality of instruction in basal reading programs? 
The data from our four sites suggest that the committees 
perceived the booklets as useful, and that the booklets 
contributed to their knowledge about reading. How and if this 
knowledge will be conveyed to publishers will be the topic of 
another study (and our next paper). Our sense is that the 
teachers on the committees in our study became not only more 
knowledgeable, but also showed signs of becoming more demanding. 
Our hope is that this will lead publishers to attend to the 
issues raised in the booklets. 



30 



Adoption Guidelines Project - 30 
References 

Anderson, R. C, Osborn, J., & Tierney, R. J. (Eds.) (1984). 
Learning to read in American schools . Hillsdale, NJ: 
Erlbaum. 

Austin, M., & Morrison, C. (1983). The first R, The H arva rd report 

on reading in elementary schools. New York: MacMillan. 
Borko, H., Eisenhart, M., Kello, M., Vandett, N. (1984). 

Teachers as decision makers versus technicians. In J. A. 

Niles & L. A. Harris (Eds.), Changin g perspectives on 

research in reading/language processing and instruction 

(pp. 124-131). Rochester, NY: National Reading Conference. 
Brophy, J. (1982). How teachers influence what is taught and 

learned in classrooms. Elementary School Journal, 83, 1-13.- 
Comas, J. (1983). Item analysis: Basal reading evaluation 

form^. Unpublished manuscript, Indiana University. 
Courtland, M. C, Farr, R.. Harris, P., Tarr, J. R.. & Treece. 

L. J. (1983). A case study of the Indiana state reading 

textbook adoption process . Unpublished manuscript, 

Bloomington , Indiana . 
Dole, J. A., Rogers, T., & Osborn. J. (in preparation). Improving 

the textbook selection process: Case studies of the 

adoption guidelines project . 
Duffy, G., & Mclntyre, L. (1982). A naturalistic study of 

instructional assistance in primary gradereading. 

Elementary School Journal, 83, 15-23. 



31 



Adoption Guidelines Project - 31 

Durkin, D. (1978-79). What classrooms observations reveal 

about reading comprehension instruction. Reading Research 
Quarterly , 14, 481-533. 

Durkin, D. (1983). Is there a match between what elementary 

teachers do and what basal reader manuals recommend? The 
Reading Teacher , 37, 734-744. 

Farr, R., & Tulley, M. A. (1985). Do adoption committees 
perpetuate mediocre textbooks? Kappan. 66, 467-471. 

Farr, R., Tulley, M., & Rayford, u (1984). Local district 
adoption practices in the nonadoption states , mimeo, 
Bloomington, IN: Indiana University. 

Fisher, C. W., Berliner, D., Filhy, N., Marliave, R. , Cohen, L., 
Dishaw, M., & Moore, J. (1978). Teaching and learning in 
elementary schools: A summary of the beginning: teacher 
evaluation study . San Francisco, CA: Far West Regional 
Laboratory for Educational Research and Development. 

Hodges, C. A. (1980). Reading comprehension instruction in the 

fourth grade: Three data- gathering methods. In M. K. Kamil 
& A. J. Moe (Eds.). Perspectives on reading research and 
instruction (pp. 110-116). Washington, DC: National 
Reading Conference, Inc. 

Hoffman, J., & O'Neal, S. (1984). Curriculum decision making and 
the beginning teacher of reading in the elementary 
classroom. In J. A. Niles & L. A. Harris (Eds.), Changing 
perspectives in research in reading/language processing and 
instruction (pp. 137-145). Rochester, NY: National Reading 
Conference . 



32 



Adoption Guidelines Project - 32 

Mason, J., & Osborn, J. (1982). When do children begin "reading 
to learn?" A survey of classroom reading instruction 
practices in grades two through five (Tech. Rep. No. 
261). Urbana: University of Illinois, Center for the Study 
of Reading. 

Osboru, J., Wilson, P. T., & Anderson, R. C. (Eds.) (1985). ' 

Reading education : Foundations for a literate America . 

Lexington, MA: D. C. Heath. 
Powell. D. A. (1985). Retrospective case studies of individual 

and group decision making in district-level elementary 

reading textbook selection . Unpublished dissertation, 

Indiana University. 
Resnick, D. P., & Resnick, L. B. (1985). Standards, curriculum, 

and performance: A historical and comparative perspective. 

Educational Researcher, 14, 5-20. 
Shannon, P. (1983). The use of commerical reading materials in 

American elementary schools. Reading Research Quarterly , 

19, 68-85. 

Tulley, M. A. (1983). A descriptive study of the intents of 
selected State -level textbook adoption policies . 
Unpublished Doctoral Dissertation, Indiana University. 

Woodward, A, (1986). Over-programmed materials: Taking the 

teacher out of the teaching. American Education , 10, 26-31. 



33 



Adoption Guidelines Project - 33 
Footnote 

••■The reports of the first meeting are contained in the book, 
Learning to Reading in American Schools , edited by Anderson, Osbom 
and Tierney. A second book, Reading Education: Foundations for a 
Literate America, edited by Osbom, Wilson, and Anderson also 
contains a number of articles that pertain to basal reading 
programs. 
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Table 1 

The Organization of the Booklets 



Title of 
Section 



What You Will Find 
in the Section 



How You Can Use 
The Section 



Research and 
Practice 



-discussion of important 
background information 
about the topic of the 
booklet. 

■current thinking drawn 
from theory and research 

-instructional implica- 
tions for basal reading 
programs . 



-to develop an under- 
standing of how the 
topic is typically 
treated in basal 
reading programs . 

-to learn about 
current research 
and thinking on 
the topic. 



Guidelines 



-statements telling what 
to look for when evalu- 
ating the topic. 



-to get a quick over- 
view of the most 
important information 
in the booklet. 



Worksheets 



-worksheets with specific 
directions for how to 
evaluate the topic. 



-to obtain specific 
guidance for evalu- 
ating the content 
and quality of the 
topic in teacher's 
manuals , textbooks , 
and workbooks. 



References 



-references for original 
studies and additional 
reading on the topic. 



-to find out more 
about the topic. 
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Table 2 

GUIDELINES FOR EVALUATING VOCABULARY INSTRUCTION 



When a basal program uses a definitional approach , look for: 
(see page 7) 

• words that name known concepts, or 

• words that are easy to define 

• words students are likely to be familiar with used in 
the definitions; and sometimes 

• example sentences , and 

• specific suggestions for discussing definitions 
When a basal program uses a contextual approach , look for: 

• instructional contexts : in which sentences contain 
information that permits only accurate interpretations 
of unknown words. (see page 8) 

• natural contexts: in which specific guidance is given 
to teachers for helping students derive word meanings, 
(see page 9) 

When a basal program uses a conceptual approach , look for: 
(see page 12) 

• words that are crucial to an understanding of the 
selection 

• instruction that establishes ties between new and 
related words, and what students already know 

• instruction that utilizes new words in a variety of 
sentences and selections 

Look for the extent to which particularly useful words are 
maintained across lessons. (see page 13) 

Look for vocabulary instruction word lists that contain 
reasonable choices for the students using the program. (see 
page 14) 

Look for clear directions to teachers, clear instructions to 
students and sufficient information in the manual for 
teaching the lesson. 
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Table 3 



Publisher: 



Grade Level; 



Prograio: 



VOCABUURY INSIRUCTION 
WORKSHEET 

Copyright 



Lcuson; 



Evaluator: 



TARGET LESSON 



Initial Reading 



Key Words 



Teacher Manual Words 



Overlapping Words 



Workbook/Skill Sheets 



PRIOR LESSON 



FOLLOW-UP LESSON 



Rating; 



pp. 



PP- 



pp. 







Which words are introduced 


Comments on Activities 


Comments on Activities/Workbook/ 






Approach 


through this approach? 


Tcacher*s Manual 
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context 










CUIDELINES 
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